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INTRODUCTION 

This workbook Is designed for use In a workshop or Independently hy 
^ callege or university group organized to revise a faculty evaluation 
system. We presume that» unless the particular Institution was organized 
yesterday y some form of evaluation already exists. It may not be satis- 
factory or It may be quite good^ needing only slight revision. You may 
find the individual chapters » forms and appendices that follow more or 
less helpful. Ve trust that you will make Judgments about how muct^ time 
you need to spend on each section and what Issues are moSC CtlElcal for 
your setting. - ; ^ 

We have written this workbook primarily for faculty and admlnis- • 
trators who will be making decisions ^^bout the design and implementation 
of an evaliiation system. Many colleges and universities are discovering 
a need for more formal evaluation, greater emphasis on institutional 
rather than individual professional needs and public accountability of 
those who make personnel decisions. Some of the factors that demand 
better evalxiation systems |^e unfortunate: lessened faculty mobility, 
enrollment ^^iecireases and financial stringency. Other signs are more 
encouraging: an Interest in more humane and fair reward systems and the 
increasing popularity of practical and effective, faculty development 
programs. Hopefully, your system will be revised both because it needs 
tc be and because you believe the institution will be a happier and more 

productive place to work witfi a better, evaluation program* 

m 

The workbook is divided into chapters wh^ch outline sequential 
steps to making decisions about a system for faculty evaluation. Following 
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««ch chapter la a aeriee of queaclona to be anavercd- by a group within a 
college or unlveralty before proceeding to the next stage. These questions 
may serve as the backbone of a weekend workshop on faculty evaluation or 
may be used throughout the process of revising a faculty evaluation 
system. * . < 

The authors of this workbook make sotne basic assumptions about 
evaluation. We believe that the evaluation system must be compatible 
with the formally stated college mission and the operational style of 
the InstitOtion. We assert that there should be public discussion of the 
philosophy and purposes of evaluation. We believe that evaluation systems 
need to be both thorough and sensitive, but also easy to maintain. We 
believe that those who are to be evaluated must be involved in the * 
design, lnq>lementation and review of the process. We think" that everyone 
who is served by faculty should have a hand in evaluation — students, 
of course, but also colleagues, the administration and significant 
others, perhaps including Trustees and some people who are external to 
the institution. On the other hand, we think there is no need and little 
-Justification for proceeding as though no one else had ever addressed 
the evaluation problem before. Any system must fit your peculiar situ- 
ation; it shoifld also be based on the best research and development that 
has gone on elsewhere. Finally, because evaluation is such a complex 
process, we think that no one approach or method is perfect or adequate. 
Multiple sources of data and a variety of approaches to making evaluation 
decisions are the greatest safeguard in designing an effective systemi " 

Based on the literature and our own experiences, we have tried to 
suggest an orderly process, for reviewing and revising a faculty evaluation 
system. We have -consciously skipped over issues that do not seem to us 



to be at the center of the review process. However » we recognize that 
mone of these popular topics will need to be discussed In any workshop . 
or by any group which approaches evaluation redesign. Therefore, In the 
appendices we have Included sections on student rating Instruxnencs, 
differential weighting of faculty activities and growth contracts. As 
these Issues arise, the appendices may prove to be helpful guides for 
discussion.. However, these topics themselves are not as. critical as Is 
an orderly review of the process of revising an evaluation system. They 
are merely elaborations on techniques that may need to be considered. 

We do not presume that any group proceeding through this workbook 
will end up with an evaluation system proposal Identical to that produced 
by any other group. But the differences should reflect the differences 
between Institutions, not a lack of consideration of generic Issues. 
Completion of this workbook and the redesign of a system should be seen 
as one stage In the development of an evaluation process. Continual 
review and revision Is what will keep the evaluation system alive and 
well* 



Joan North 

Director of Faculty and Staff Development 
Small College Consortium 4 
Washington, D.C. 

Stephen Scholl 

Dean of Educational Services 
Ohio Wesleyan University 
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II. 

REVIEWING AN EVALUATION SYSTEM 

A h«lpful approach to rcvlalng an axlatlng faculty evaluation 
pr6ca«« la to aaaume and act aa If you ara beginning fron acratch. Aak 
bitelc quaatlona, auch aa: Who la aaklng for fa9mty evaluation? Who will ' 
want to aee the reaulta of evaluation? Who iV^JLntereated in the proceaa? 
The inltiatlvea to review a progran (or begin one In the caae where none 
•xlata) may eoanate from exlatlng rcpreacntatlve groupa» auch aa faculty, 
atudenta, the administration, even parenta, truateea or legislators. Who 
!• Interested and who they represent are important Issues. If there Is 
not some agreement among a significant set of groups that something 
needs to ^>e done or changed, there Is little point in proceeding any 
further. 

Once the decision to review has been made, ^e process of revising 
an evaluation program can be divided into phases^ focusing in turn on 

y 

study, adoption, I m pl eae ntAtlon, evaluation and review once again* In 
this chapter we will focus most of our attention on the study phase. 

Stiidy Phase 

The formation of a well-structured study group is critical. It 
should be neither too small nor too large; -slx-to-nine participants is 
workable. Althoiigh those to be evaluated nnjst be the most significant 
participants, at least three constituencies are usually needed in a 
typical college or university — faculty, administration and students. 
Sometimes members of an institution's, governing board may also be 
Interested in working with this group. 
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Th« study group's objective* should include high visibility, opsn 
snd broi^d coinaunliji|tion , credibility (who is on the cocBBlttse and how 
th«y art •elected are the mott Important Initial factors In catabllahlng 
credibility), the gathering of extensive data and thorough undcratandlnR 
(reading « vorkahopa^ etc.) of laportant evaluation issues. 

Most study groups work wore effectively if they use a standard 
group problem-solving process.^ The following is a typical process: 

1. Define and get agreement within the group on the problem (spend 
sufficient time at this stage or all that follows will be ineffi- 
cient) . 

2. Set goals (statements that arc the reverse of the identified prob- 

. lea) — what should the situation be when the process is completed? 

3. Develop and consider several alternative solutions. Including, 
perhaps, maintenance of the status quo. 

i. Analyze the forces for and against change (technically, "force- 
field analysis") on paper and concentrate attention on forces chat 
reaiat change rather than those which promote change (the latter 
are already on your •side). 



For more detailed guidance on group problem solving ske William Morris 
and Marshall Sashkln, "Phases of Integrated Problem Solving," in Pfelfer 
and Jone^ eds. , The 1978 Annual Handbook for Group Facilitators 
(La Jolla, Cal.: University Associates, Inc., 1978), pp. 105-116. For 
alternate techniques, see Arthur Chlckerlng,^ al^. Developing the 
College Curriculum . Appendix C ("Planning Tools"), (Washington, D.C. : 
Council for the Advancement of Small Colleges, 1977), 279-290; 



5. Plan a strategy, including auggaaclcm^ ragardlng %rhaCvto do. by 
whom and vhan. Put the atrat^y In draft fora and dlacributc Id 
aaidtig the major conatlcucnclca of the InatitOtlon, perhapa aepa-* 
rating aubatantlva laauea from concama with the evaluation proceaa, 

6. Provide for at leaat one public campus dlacuaalon of revlalon 
propoaala at thla "draft** atage. At auch -^^ng etrategy Itaelf 
la Important — for^^tance^ a good vleuax model , auch aa an over- 
head projection or large chart focuaea the attention of the group 
up fronts together^ and Indlcatea a common search for solutions 
rather than tangential critiques of minor aspects of the process • 

7. Make reccmnendatlons^ incorporating suggestions from the campus 
meeting and Including means for periodic revision of the system. 

Any small group that is charged with wrestling through the problems 
of evaluation Is likely to face some tensions In its own funr*^ { :ning. 
The gro^ is in the spotlight; It must produce recommendations that 
directly Impact on the livelihood and well being of Mry people. Purther. 
academic committees are not famous for efficiency* teamwork or decisive 
action. So the study group ought to attend to ±xs internal pracpQS of 

decision-making and taking action. Technical advice abounds on small- 

2 

group' effectiveness. At minimum, the group should begin by deciding how 



Soae eo^amples of straightforward primers on effective small group work 
are: "Decision-Making/* in William- Bergquist and Steven Phillips, A 
Handbook for Faculty Development , Vol. I (Washington, D.C.: Council 
for the Advtacement of Small Colleges, 1975), pp^ 157-16A;, and Andre 
Delbecq* Andrew H. Van de Ven and David H. Gustafson, Group Techniques 
for Program Planning (Glenview, 111.: Scott, Foresman and Co., 1975). 



to d«cldc (voclng, cofMi«n«u«, or Home Interactive procr««) 4i{%d how ro 
act (by delegation through a ataft, by aharlng aaalgniDenta • through a 
chalrperaon. etc • } . 

The entire atudy and recoonendat Ion proceaa will take about five- 
to-nlne i&ontha under normal clrcumatancefi , with Inplemcntatlon uaually 
ccolng in the following academic year. If the atudy proceaa Itaclf takea 
aK>re than a aingle academic year, energy for the iaaue tends to wane «nd 
people begin to believe that nothing will happen. A fairly tight time 
achedule» with apecial meetlnga for dlacuaaion of the revision of the 
evaluation program, is much to be deaircd. 

There are several comnon sense detaila to be observed during the 
atudy proceaa. For inatance, it is wise for the study group to share its 
thoughts with smaller groups of faculty informally prior to circulating 
a general draft proposal and having a large campus meeting* It is usually 
advisable not to have the general campus meeting at a regular faculty 
meeting, to avoid the notion that an early decision is going to be 
forced and In order to have it truly perceived as a time for stydy and 
suggestions rather than legislation. It is important that the full study 
group participate in any campus meetings to show general support for the 
proposals and to be available to answer any specific suggestions chat 
may. represent differing perspectives. Such participation continues to 
enhance credibility for the study process. 

Other Phases 

During the adoption phase those to be evaluated, the faculty, 
should be able to thoroughly analyze recommendations and formally accept 
ownership of the evaluation system^ Lf they cannot be convinced that 



this Is the best way to' go for them. It Is b^ter to revlS^ r^ommenda^'^ 
tlons than to force an unacceptable program. During the Imp lemen tat Ion ^ 
phas,e « the critical eleme&t Is a smoothly functioning process that Is 
perceived as thorough and yet does not take so much time that It becomes 
burdensome. Again, careful planning and execution are Important. The 
evalxiatlon phase Is an obvloiis tribute to a belief In evaluation of all 
Important enterprises In the Institution. There Is always something that 
can be Improved in any process. Finally, It sl^ly must be assxjmed that 
any new program may in turn be revised again, so building in a time for 
formal review is only good planning* 

. In Form 1, which follows, are a series of questions pertaining to 
the creation of the study process^The answers to these questions should 
provide you wlCh the basic action planning design for reviewing faculty 
evaluations. 



/ FORM 1 . 

Determining the "Study Process 

/ 




Rho Initiated this review? 



Who will the ^udy. group be? How Is it to be chosen? What groups 
and how many from each groxxp are to be representative? 



To whom is the study^ group responsible^ 



How will the study group keep the faculty informed? 



V. 



How will the study group operate in makings decisions and taking 
action? - 



\ 



How will a final declsloti to adopt and Implement a revised evalu- ; 
ation process be made?- * ■ 



^What is the general list of. activities for the study and lecommen- - 
datlon process?' ^ _ ' ' . . - > : 



Chart the important steps to be used in the study process; give 
'completion deadlines. ^ . - » 



Consider using this completed work sheet for a check on the process 
midway through' t^ie study (especially -jtist prior to any general i 
4iampus meeting called to review draft recommendations). The study 
group should ask itself: Has anything changed since we began? Are 
we on schedule? If there are revisions or prdblems in the process > 
are they being- appropriately dealt with? Are we ftmctioning effec- 
tively as/k group? Finally, review this sheets just prior to propos- 
ing a revised evaluation process fbr adoption. Ask the sasie questions 
that were asked at the midpoint. Checking the process in this way 
can' prevent unfortunate surprises at ctiticai stages. - 
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III. 

EVALUATION PHILOSOPmrffliD THE NATURE OF THE COLLEGE 



lough few people pause to exflrmlne an ev|J.uatlon philosophy, the 
consideration Is helpful for ensuring ^congruence between evaluation 
practices and the nattxre of a particular institution. If the system of 

y 

evaluation is incompatible with the college mission and style, it. will 
not be operational. Imagine, fpr Instance, the difficulties for a small ^ 
college which emphasizes community and individual growth for its faculty 
and students while it employs a competitive-, comparative evaluation' 
system* 

s. . . 

This chapter outlines four generalized evaluation philosophies. The 
approaches overlap and they are admittedly Ideal types. Discussion of 
these types may help to clarify a study group's philosophical approach 
to* the. problem. The next chapter will focus on more specific purposes or 
functions of evaluation.. ^""^ 

1.. Sinrnn^tlve comparison with other faculty . Comparisons can be made 
within the college, within a department, with faculty nationally, 
within a . discipline nationally, with faculty of the same rank,, or 
with faculty having similar classes In the school or nationally* 
MosJt faculty evaluation systems have traditionally relied on such 
comparisons and resembled periodic "exams", with the tenure decision 
perceived as the "final"* It Is understandable that die faculty 
niember should ask, "With whom am I being compared, and are there 
any differences In dur ^ituatlons which may work against me?" 
Summatlve comparisons often employ quantification, but this approach 
does not require it^ \ 



Advantages : It is sometimes easier to Judge people in 
comparison to other people than to measure them against an 
Ideal too often not. clearly d[efined» such as "effective 
teaching." So, this approach can provide relatively ohject^lve 
comparisons and can help to' make difficult administrative 
decisiona by relying on differences between performance and. 
- ~ production of various individuals. When resources are scarce, 
there can be a' ranked list of people to be rewarded,, promoted 
^. or dismissed. One ends up with the cream of the crop. 

Disadvantages : The discrimination between individuals may* 
appear to be so slight that ^cisions are not deJEezisible. 
There is certainty only about who are the best and worst 
faculty. This type of aummative evaluation generally comes at 
the end of a semester or year, and the burden of proof is on 
the instructor. Individuals may be compared to inappropriate 
groups, for example, to someone in an institution with more 
support for research or to someone with smaller - classes in a . 

more popular subject. The approach forces a ranking, and 

^ ft . 

implies that some faculty must be performing poorly even if 
the differences are slight. The bell curve is not always 
* appropriate.^ 

Criterion-based evaluation , ^is philosophy assumes that there is 
an ideal^ standard or a set of basic criteria against which' faculty 
are/ Judged. When the criteria are explicit, they may include a 
f^ed number of publications or a certain score on^ student evalu- 
ations or 'other .performance measures. Often, however, the . ideal is 
only implicit in personnel decisions and not explicitly publicized. 



The **ldeal standard" may be a "lir. Chips" type, the internationally 
renowned scholar, or the imoffldal coiqposlte of favored tenured 
professors* . " . 

Advantages ; If used ejq>licitly, this philosophy promotes . 
^discussion about 'ideals or criteria which may result in tjseful 
. institutional standards and can ^ therefore, provide for qtiallty 
contlrol. Explicit criteria also provide clear giildellnes for 
faculty whq are able and willing to mold their behavior to fit 
th^ stmdard\ 

Disadvay qfees ; This philosophy may limljt tolerance for 
Individual differences.. The philosophy may lead to-a trait 
approadh to evaluation, rather than looking at behavior or 
student, learning outcomes. Thus it may have a tendency to 
promote "looking right" and effxisive game-play dLng. If, on the 
other hand, the criteria are not explicit, faculty may be 
evaluated on evidence ^about which they are Ignorant* 
Developmental or formative evaluation for growth. This approach 
emphasizes the individual and personal" growth, rather than a. 
relative ranking compared to others or to an ideal standard* A 
"growth plan" philosophy implies that evieryone can and should 
improve in performance and the change, rather than the relative 
strength, is what is rewarded* Colleges recruit good faculty who 
show promise of continued growth* Faculty should understand clearly 
and accept what needs to be done during, the evaluation period. 

Advantages : Conq^arison is with oneself and is related, 
directly to one's professional growth. Evaluation frequently 
begins at the start of a semester, year or several-year period 



allowliig time to change and In^rove. This approach may stimulate 
more productive effort than the summatlve comparative approach 
aq4 may produce mor^^ behavior change. Development and evaluation 
.may be combined In- this approach »^ Multiple sources of data are 
clearly required* The process Is highly Individualized. 

Dlsatlvantages : Developmental evaluation takes more t ime 
because each person mtist be dealt with at leogtfa^, although not ' 
necessarily every term or year. If an Indlvldtial's' personal 
growth plan Is to benefit the department or college, consider- 
' ablfe customizing and planning are needed early In the process. 
It may be difficult to establish and'malntaln equitable In- 
dividual expectations across an entire Institution. Even with 
Improvement, a person may still be performing so pooxly that, 
the college wishes to terminate him, but the developmental 
approach has- not been ^used often to support dismissal. There 
can be an Implied "contractual" c^bllgatlon to retain everyone, 
though Institutional needs' may change during the period of 
evaluation. 

Objectives-based evaluation . This approach is borrowed from the 
military and business world, where it is usually labeled "Manage-, 
m^nt by Qbjectlves". This philosophy holds that the creation of 
annual- individual measurable objectives will lead to explicit 
results. To be effective, the entire institution must be , involved,, 
from top to bottom. Thi^ philosophy essentially suggests a contract 
approach that "if fulfilled, results in some Individual reward." 
The usual difference between this philosopby and the developmental 
approach is^hat the objectives established for indivicjuals are 
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\ ' ■ * 
based^^prlmarily on the institution's needs, not the individual's, 

g Results, rather than process, are most .valued. ^ 

Advantages ; This philosophy clearly establishes the 

^ primacy of the institution over individuals within the college. 

It requires that individuals agree ^on institutional objectives 

and their own responsibilities for helping to achieve them. It 

forces the college to beN:lear about its objectives. It is 

systematic, reasonably predictable and may provide a unifying 

force within the institution. ^ 

Disadvantages : This philosophy involves the creation of 

periodic goal-setting and review sessions with all college , 

personnel. Thus, it is t^^ine-^consumlng. Becaiise 'this approach 

so much tied to institutional objectives. It may ignore 

individual needs or stifle creativity. Also, it may not be 

possible to objectify the most valued goals of institutional 

or individual life. Not every faculty activ^^6y can be derived 

from the college mission statement. 

The form following this chapter asks you to choose which evaluation 

philosophy most closely matches your Insti^tutionT 




/ 
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POBM 2 • 

Bvaltiation Philosophy and the College Mission 

■ r 

Consider your college U ad^jsion (fonaally stated), general opiating 
style and attitude toward people. Based on vfaat you perceive, dj^l^ct the 
•evaluatioa philosophy vhich you believe should be most appropriate for 
your icoUege, and exp lain why you believe it fits. Suggest any is^rtant 
inodifications in the approach vhich night be necessary. 

1. Effective faculty evaluation compares a person's performance with 

others' performance to determine who is performing better at critical 
Junctures (summative comparisons) • 



i 



f 



I 



,2. Effective evaluation measures a person's performance against specific 
criteria or an ideal standard (criterion-based evaluation) . 



Effective faculty evaluation measures a person's improvement in 
performance periodically (developmental/formative evaluation) . 

V ^' - 



Effective evaluation measures regularly the results a person achieves 
agalxtst predetermined objectives (cbj'ectlves-^based evaluation) .' 



Other, 



/ 

/ 





. ^ PURPOSES OF FACDLTT EVALUATIOl 

HI' 

\ • 

Ho natter vhat the institutional phllosophjfT aost evaluations have 
at least tvo isipllclt purposes — to naalce judgments about 3n activity so 
•it may be. appropriately valued and revards may be distributed, and to 
improve the activity Itself. One of the basic problems about faculty 
evaluation systems is confusion about using evaluation for personnel 
decisions and/or for faculty development. While evaluation data is 
almost always used ^or personnel decisions, the types o^ information 
gathered may be more appropriate for developmental purposes. For instance, 
many student evaluations of Instruction contain detailed questions about 
specific teacher behaviors as -well as summary statements about the 
overall effectiveness of a faculty toember and a course. Probably onlv 
phe 8\mmiary statements about the teacher are useful in making key per- 
sonnel decisions, while the detailed observations of behavior and ^ 
Ascriptions of the course can be better used for improving teaching 
practices. Further, It is often forgotten that within a particular 
Institution there are usually at least * two -classes of faculty — the 
tenured and the untenured. Evaluation philoso^y is likely to- differ in 
the -eyes^of those two classes if the purposes and- assumptions about the 
consequences of evaluation differ. Even if all members of. the faculty 
participate in the same evaluation system, the ultimate consequences of 
a poor or an outstanding evaluation are likely to be substantially 
different for tenured and untenured members. Hence, there are differences 
in perceived purposes. Sorting out these issues and the kind of data 
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needed lllustretc why aa earl^ concern In revising the evaluation 
pi^ceaa^vuat^ have to do with defining the purposes of the evaluation. * 

The question of whether a slngleir faculty evaluatjtoiv^ystein can 
serve both' personnel . decision-making and factilty development needs to be 
resolved. One current view Is that it Is necessary to separate the two 
functions Into separate evaluation processes. While most would agree 
that In some Ideal conditions both functions ought to be coiAlned and be 
mutiially supportive, such coii4>±natlon can be very difficult to accom- 
plish because of the different purposes being served. For personnel 
decision^ jiromparlsons are often made» usually across can^us and usually 
using comparative data to help make difficult decisions that result In 
significant rewards or punishments. A primary goal of such a system Is 
to be perceived as fair and legally defensible. Evaluation for develop- 
ment Is more frequently based on measuring change In Individual per- 
formance and Is most often voluntary rather than compulsory. Change Is 
often most rapid when a faculty member discovers that something he or 
she thought was ^golng well turns out to be perceived by others as an 
area needing Improvement. To discover this requires taking risks in 
evaluation. Assessment for development needs to encourage experimentation 
and be tolerant of failure. If the same evaluation system and the same 
instruments are used for both personnel decision-making aiid development, 
voluntary risk-taking and experimentation are less likely to be forth- 
coming. 

Admitting these practical difficulties, it is sometimes argued that 
the separation of personnel decision-making from development weakens 
both activities. Even if it is not possible to bring both functions 
together in a single, integrated, system, it may be desirable to bring 
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them closer together. \t±ll faculty, pursue development ectlvlclcs in a 
; ayatem totally segregated from the personnel rcvard structure? Though 
faculty probably Improve their performance primarily because of an 
intrinsic drive for excellence and professional pride, external rein- 
forcement for improvement must not be overlooked. Some claim that in a 
progrsm of faculty development separated from the reward structure the 
only teachers who participate are those who need development least. 
There are ways to utilize developmental evaluation during lengthy 
periods (two-to-three years?) of professional growth and then review 
thbse evaluations for personnel decision-making only when key decisions 
must be made, thus meshing the two systems fairly effectively. Different 
people in the- faculty and administration may be concentrating their 
efforts on development or the personnel system, but they should have 
means of cOTOunicating with and supporting each other while still main- 
taining a sense of fairness in the process. 

Beyond the basic functions of personnel decisions and professional 
improvement, faculty evaluation may serve additional explicit or implicit 
goals* Accrediting agencies often require explicit forms of evaluation. 
Students want to have influence in evaluating teachers, and they want 
information about courses and teaching styles that go beyond catalog 
descriptions. Sometimes an evaluation is used just to provide a facade 
of objective accountability. 

The next exercise should help the study group clarify the single or 
multiple purposes evaluation is to fulfill. The philosophy of evaluation 
discussed in the previous chapter should provide the basic approach to. 
serving these varied purposes, but it does not necessarily determine 
which functions will be served. 



* • ■ # 
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F08M 3 

\ 

Purpo»M of PActaty Ev«lu«tlo& 

-Choose OM or aor« of the folloirlns option*. If oiore than one option i« 
cboecn, rank your choices In term of prljucy of porpoM. 

1. To provide ccnperetlve dete about faculty performance upon 
ifhlch personnel decisions vlU be made. 



2* To^ provide data on an Individual faculty 

performance (not compared to others) upon which 
personnel decisions vlU be made. 



3. 



To provide caaparatlve data about faculty performance 
^A±^f:ir±ll provide a baals to motivate Indlvldtials to 
Improve their performance.. 



4. To provide data on an Individual faculty member's 
' performance (not compared to others) from whlxh 

Individual faculty members will be motivated to 
Improv^^^ 

5. To es t ^> A^h base- line data about an Individual's 
performance^ against which the Individual's growth and 
development will be evaluated for personnel decisions. 

6. To provide Information for college--wiae faculty develop- 
ment efforts. 

?• To give stiidcnts a means of registering their views on 
faculty performance. 



8. 



To provide students with consumer Information for course 
choice. 



9. To conform %rlth accreditation requirements. 

10. To Inform the Institutional governing board of the 
quality of faculty performance. 

11. To provide data to move from an arbitrary or intuitive 
I>ersonnel system to an information-based system. 

12. To eliminate undesirable or unproductive faculty members. 

13. To reward and advance the most desirable and productive 
faculty members. 

14. Other purpose: (fill in your own) 
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V. 

IDENTIFYIII& Ain>- WEIGHrHiG FACULTY AC T IVI T IES 



Is every -faculty member expected to perform every facet of faculty 
work equally well? Is every faculty member doing the same taxings, spend- 
ing equal time and effort doing them? Not likely, nor Is It likely that" 
any Institution would wish It so. 
. Faculty activities most often formally evaluated are teaching and, 
•for most^'coftleges, schtolattrly or creative production. The faculty perform 
a variety of additional fractions which are expected by the college, but 
seldom are part of a formal evaluation system. Thus. It should be no 
wonder that faculty spend little time advising or contribute llttl^ to a 
committee in -preference for other klpd» of activities. If the college 
places a value on faculty activities in additira to teaching, those 
activities shotjld be evaluated and rewarded. 

Most evaluation, systems assign relative: welghts^^o different ac- ' « 
tivltles, though often only implicitly and sometimes in esoteric and 
dysfunctional ways. Evaluation may be improved by 1) making the relative 
value or weight of key activities (such as teaching, research and campus 
service) explicit, 2) allowing differential weighting of activities to 
maximize rewards for peculiar strengths among faculty, and 3) allowing a 
variety of activities to be evaluat?j^ and rewarded beyond the basic ^ . 
core. Such variations and refinements are possible no matter which 
•philosophy of evaluation is followed, thougji the "sximmative comparison" 
approach, if extreme, will limit the freedom to use differential weight- 
ing, y 



Form 4 provides an opportunity, to Identify and weight faculty , 
activities to be evaluated. 



FORM 4 



Faculty Activities to be Evaluated and Weighted 



Thai determination about ^ich activities are rewarded can be made college 
vide, or by department or by individual negotiation. The Important 
factor is that the faculty understand and acdept the list and the rela- 
tive value placed on different activities. Below la a list of activities 
In which faculty may engage and for which they may be rewarded. 

First, check those activities which are/should be evaluated in your 
college, then place a percentage rating (or range or percentages) on 
those you checked. ,Once agreement is reached on this phase, you have the 
beginning of an explicit, weighted evaluation &ystem. 

4 ■ 

Is Should be 

Rewarded Rewarded Weighting 

1. Teaching — general 

a. Pedagogic methods 



b. Teaching skill 



c. Subject-matter competence 

d. Subject-matter difficulty 

e. Grading' practices 



f« Student learning outcomes 
g. Student satisfaction 



h« Enrollment In courses 



1. Other aspects 



j . Other aspects 



2, A^ylslng — general 



a. Number of advisees 



lb* Advisee retention 

c. Advisee job and graduate 

placement 

d. Advisee satisfaction 



e. Other 



Other 



Is Should be ^ 

Rewarded Rewarded ^ Weight In^ 



College service — general 

a. Assisting other faculty 

b« Effective participation in 
committees /. 

c* Assisting in student recruitment 

d. Assisting in fund raising 

e» Succefp In grantsmanshlp 

f • Relationships with colleagues 

g. Other ^ 

h. Other 



Community service — general 

a. Effective participation in 

coammnlty affairs 

b. Community leadership 

c . Other 



Professional grox/th — general 
a« . Degree completion 

b. Additional study 

c. Professional publications/ 

research/ creative work 

d. Participation in professional 

meetings 

e« Leadership in- professional 
associations 

f. Participation in curriculum/ 

teaching institutes, 

workshops 

^ > - 

g. Design of new courses/ 

curricu^txm . 



!• Should 
Rcvarded Rwrded WeightinRj^ 



0 



Use of new methodologies In 
teachings advising or 
research 



i. Other 
J . Other 



Administrative duties — general 

a. Committee leadership 

b. Part-time administrative 

assigxments ^ 

. c. Department /divisional chair 

d. Other 

e* Other 



7. Personal traits — general 

a. Creativity *t 

Friendliness . - 

c. Religious /moral commitment 
d« Coo'perativenesa 

e. . Intelligence 

f. Loyalty to institution 

g. Other 

9 - 

h. . • ^ Other 

8. Miscellaneous 

a. Competing job offers 

b. Length . of service and rank 

c. Other j_ 

d. Other 
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. VI. 
^ USING DATA TO SUPPORT EVALUATION 



I •• 

li^ ' ■ ■ ■ 

Once activities to be evaluated have been identified and weighted. 
It Is possible to suggest types of data which might be collected. Gener* 
ally 9^ evaluartlon vlll^>e ^more valid and fair if more than one" kind of 
evidence Iflf used. The only caveat to be note^ about using many sources' 
of information Is that the data must be limited to that which can be) 
easily collected and analyzed. Make the system practical or it will soon 
break down. 

Preliminary to. the various forms and systems of gathering observa- 
tions and Judgments^ there is a need for basic documentation for eyalu- 
ation. You might consider a kind of archive of regularly collected 
faculty-supplied- or administration-supplied evidence* such as self- 
reports of activity, syllabi, tests and course materials, copied of 
publications, professional programs and educational and honorific docu- 
ments, course enrollment data, advisee assignment data, placement records 
of students and so bn« Much of this "raw" data might be routinely filed 
at convenient intervals well ahead of the scheduled" times for gathering 
ofchers* views and analyzing the corpus of evidence. The file of such ^ 
data should be open review by the faculty member* 

Since one common denominator among faculty is teaching, it is 
probable that one type of evidence will be some kind of student evalu- 
ation of teaching. Instrtiments for this kind of appraisal abound, many 
of them useful in a variety of colleges* With the voltmilnous research on 
the validity and reliability of student evaluations, there can be little 
excuse for not iising well-designed student ratings (for more student 



mppralMls, SM Appendix .1) • But^ using only student' evaluations for 
cyaltietlng teaching Is equally inexcusable* Students are the most re«* 
liable source of Information about a teacher's behavlj^ their 



ictlons to It; they are In a poor position* to evaluate course content. 



the professor's g;rowth over time or^other professional activities. 

The opinions and observations of colleagues, in the long run, are 
often the tqost critical source of data for personnel decisions^. Obser- 
vation especially can also be useful for Improving performance. But 
\mlike student .rating forma, peer ratings have not enjoyed as much 
attention, from psychometrlclans or other experts in the evaluation 
field. And the formal means of recording data collected from colleagues 
and administrators are likely to more Idiosyncratically relate to a 
particular college and stage of evaltiatlve system development than our 
student -ratings. It is useful to examine ^eer rating instruments from 
[ other institutions, but you will probably need to construct your own. 

Form 5, which fellows', •^identifies elBht types taf data wjhich , might 
be used to support faculty evaltiatlon. Questionnaires/rating Instruments 
are' usually some form of student or altmmi or.* colleague evaluation of ^ 
teaching or advising. Self reports are the faculty member's ownreflec- 
tlon on her progress and activities. This type of data becomes especially 
iiiq>ortant for the faculty member to have the opportxinity to explain her 
educational goals or any special circumstances about her performance. 
D irect observation refers primarily to classroom performance observa- 
tion, but could also include committee work, ^vising or oth«: faculty 
activities. The observation could be by a colleague in a team- taught 
course, a student, ithe dean or other colleagues. Interviews of students, 
colleagues, others can provide more detailed and specific data on an 
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individual's performance and impact. Expert opinion f rom^ outa idei:fl is 
used primarily for Judging a faculty member^ s competence within his 
dlscipllnep although it could also be used in other areas. Activity 
reports are ways for faculty to docximent what they are doing or how they 
spend time, such as time studies* This data becomes crucial for faculty 
whose activities are different from the nonn* such as those with heavy 
advising, committee or project loads. Products - can Include syllabi, 
tests, publications, papers, committee minutes, art work any product . 
resulting from the individual's professional life. Institutional records 
are current and historical^ data about those aspects of the institution^ 
which directly relate to the faculty member *s work, such as enrollmeht 
figures' for the Individual's classes, attrition rates for his advisees, 
student exam performance, or placement history for his students. 

Note that several Copies of Form 5 are provided to cover each type 
of faculty activity to be evaluated,' such as advising, teaching, service 
or scholarly production. You may need to reproduce copies if your J^st 
of ■ activities is extensive. 



ERIC 
i 



* ■ 



FORM S 



Data to Support Faculty Evaluation 



TJaa ohlB of tliaia fbrmi for aach priaclpal tvpa of activity which will be 
avaluatad (auch as taachlng, raaaarch, aarvlca, ate). Spaclfy which 
typaa of data are appropriate for avaluit lag each activity, than check 
off which. will be required (preaumably a amaller number than thoae typea 
of data which mlght.be appropriate), and finally. Indicate who la to 
supply the data (students In course*, advisees, the person being evalu- 
ated , colleagues , depariafent head, dean, alumni, outside experts, etc.). 

^f-' . 



Activity to evaluate 



Types of Data 
Ques t ionnaires/rat ing 



Appropriate Required Source of Data 



B ine^ unenta 
(specify by students, collea^tites, etc.) 



Self -reports 

Direct observation of the activity 



Interviews .of students, colleagues, others 



■ — X — 

Expert opinion . from outsiders 

Activity reports (specify) 



Institutional records (enrollment figures^ 
attrition reports, placement records*, 
atudent exam performance, etc.) 



V. 



• FORMS . 
Data to Support Faculty Evaluation ^ 

_ _ !■ _) _ 

^mm imm or^airn^^ formm for aafih principal typa of activity vhich will ba 
•valuatad (puch aa taachlng, raaaarchp aarvica, atcO- Spacify which 
typaa of data are appropriate for evaluating each activity, than check 
off which will be required (preaumably a amaller nuaber Chan those typaa 
of data which might be appropriate), and finally. Indicate who la to 
supply the data (atudents in courses, adviaeea, the peraon being evalu- 
ated, colleaguea, department head, dean, alumni, outaide experta, etc.). 

Activity to be evaluated: 



Types of Data Appropriate Required Source of Data 

Questionnaires/rating dLnstruments 
(specify by students, colleagues, e#c. ) 



•Self -reports " '* ' 

Direct observation of the activity 




Interviews of students, colleagues, others 



Expertf opinion from outsiders 
Ac^vlty reports (specify) 



\ 



Institutional records (enrollment figtires, 
attrition reports, placement records, 
student exam performance^ etc J) 



FORM S 



D«t« to Support Faculty Evaluation' 



Uaa ona of thaa« fogmi for aach principal tvpa of activity which will b* 
cvaluatad (auch aa taaching, raaaarch» aarvica, ate). Spadfy which 
typaa of data ara appropriata for evaluating aach activity, than chack 
off which will ba requirad (praauaably a amallar numbar than thoaa typaa 
of data which might ba appropriata) » and finally, indicata who la to 
supply tha data (atudenta in couraaa» advlaeaa. the fiaraon being evalu- 
ated, colleaguea, department head, dean, alumni, outaide experta, etc.). 

Activity to be evaluated: 



Typea of Data Appropriate Required Source of Data 

Questionnaires /rating instruments 
(specify by students, colleaguea. etc.) 



Self-reports 

Direct obaervation of the activity 



Interviews of students, colleagues , others 



Expert opinion from outsiders 
Activity reports (specify) 



Institutional records (enrollinent figures, 
attrition reports, placement records, 
student exam performance, etc.) 



FORM 3 

Data to Support Faculty Bvalu«clon 



? ^ —eh ^TlB6l&«r tvt># of activity which i^ll b« 

•valuatad (.uch aa taachlaf, raaaarch. aarvlea. ate). Spadfy %ihich 
'I?**^? w W^P'^*'* 'oir •valuatlng aach activity, than chack 

Of f which will ba raqulrad (praauaably a aaallar nuabar than thoaa typaa 
of data which alght ba approprlata) , and finally, Indlcata who la to 
•upply tha data (atudanCa In couraaa, advlaaaa, tha paraon bains avalu- 
atad, collaaguaa, daparcaant haad, daas, Axxmxl, outalda axparta, ate). 



Activity to ba avaluatad: 



Typaa of Dat a Approprlata Raqulrad Sourca of Data 

Quaatlonnalraa /rating Inotrxaaanta* 
(apadfy by atudanta, collaaguaa, ate.) 



Self-reporta 

Dlract obaerva^lon of tha activity 



Interviews of - students, colleagues, others 



Expert opinion from ot^tslders 
Activity reports Xspeclfy) 



titutlonal records (enrollment figures, 
attrition reports, placement records, 
st>udent exam perfomanca, etc.) 



^ VII. 



SaaDULl AMD WKPOHSIBILITY rOR COLLECTING EVALUATION DATA 



Who la tolng to collect whac data/ wh«n and how? D«cl«lona abouc 
thoao loau«4 can makm or braak tha baac avaluatlon daatgn. If confldan- 
tlallty and falrnaaa ara ijhporcanc (and thay alwaya ara), tha raapon- 
l^billty for practical, wall-undaratood and raaaonabla maana of collactlng 
■vldanca nuac b« carafully plaaa4d, Malgnad and axacucad. 

Foraa 6A and 6B ara to ba utlllxad for datalllng tha raaponalblllty 
and calendar for data collactlon. Again, there are coplea of Form 6A for 
««ch major faculty activity. Tha important point here la that thorough- 
naaa suae ba Mtchad by practical conaiderationa^of clnw and anargy. and 
tha natural rhytbaa of the academic -year. Form 6B illuatratea t^t it la 
important U> review thcae naterlala to make aure that the burden of work 
la well-dlatrlbuted and vfell-tlmed. ' 

Later, when producing Instructlona for each part of , the evaluation 
eyatem. be sure^ to spell out what Is being measured or collected by whom 
for what purpose, to whom it Is being submitted and how the data will.be 
stored, used and how long it will be kept. These specifications are not 
only neceasary under the growing pressure of concern about privacy and 
protection of Individual rights in the storage of personal Information; 
they are simply fair to the people being evaluated and to chose supply- , 
Ing data. 



POtH 6A 

toapoMlbllltT for Dm CcUactlM 



Pr«cla«ly hoir 4omm thm •valuation ayatoa mkf Two sajor datalla of 
laplaMtlat an avaliUtlon ayataft Includa aaaltniof taapooalblllty for 
collactlnt tha data and daalgnatlnt collactlon tlaaa. For aach faculty 
actlvitx to ba avaluatad. fUl out thla fora Hating tha raqulrad data 
(aaauM alao that faculty say want to add aupplaaantal data). Indlcata 
. collactlng and raportlng thia infoxmat Ion. whan 

It will ba collactad and to «ibo« tha data la aant. 



EXJKPLE 

Activity: Advlalng 
Data 

Advlaaa evaluation foi 
Retfentlon ratea of advlaaea 

Reporta from dapartaent heads 



Collected By 
Advlaor 
Haglaf rar 



When 
May 

January & 
Saptaaber 



To Whos Sent 



Department head May 



Department head 

Department head 
and dean 

Dean 



Activity ; 
Date 



Collected By When 



To Whom Sent 



CoBplete thl» form for mb mmny mJot activities aa you plan to evaluate. 



wow tA 

«««poMl>lllty for Data CoU«eelon 

Fr«cl««ly how do«« ch« ovaluation uytwm «#ock7 Two Mjor datalla of 

•valtt.tloo ayaeaa lacluda aaalgniag raapooalbility for 
collactlag tbo data and daalgnacltkg collactloo timmm. for aach facu lty 
Ktl^lty to b« avalua^od. fUl out thla for* llatlnt tha raqulrad data 
OMj««a alao that faculty My want to, add aupplaaantal data). Indlcata 

for coUactlnt and raportlof chta Information, whan 
'"^l »• collactad and to whoa tha data la aanc. 



Activity: Advising 

Advls#« •valuatloQ foi 
R«t«ntloo ratM of «dvl«Ms 

« 

lUportfl froa d«pArt»cnt hmmdt 



Coll»cfd By 

Advlaor 

tofiscrar 



Wh«n 



Nay 



JADuary 6 
S«pt«flib«r 



Dapartacot head May 



Activity : 
Data 



To Whoai Saot 

Dapartmnt haad 

Dapartaant haad 
and daan 



Daan 



Collactad By 



To Whoa S*nt 



Coaplata thla fora f 



or aa 



many aajor actlvltlaa aa you plan to evaluate^ 



4C 



Responsibility, for Data Collect ioQ 



^Precisely how does the evaltiatlon, system work? Two najor details of 
U^lerctttlag^^ evaluation system Include aa^gnlng responsibility for 
.collActla^ the data and designating collection tlmes^ For each fa cttltv 
^^^tj to be evaluated, fill out -this form, listing J:he required data 
^assume also chat faoilty may want to add supplemwitS data), indicate 
5** J?,^f*^***^^^® for cqUectlng ^ reporting this information, when 
It will be collected and to whom the data is sent. " 



EXAMPLE 
^iuctivlty^ Advising 
Data 

Advisee evaluation forms 
Retention rates of'^yisees 



Collected By 

Advisor 

Registrar 



When 

January & 
September 



Reports from department heads Department head M^y 



To Whom Sent 

Department head 

Department head 
and dean 

Dean 



Activity^ 
^ta - 



Colle'cted'By When 



To Whom- Sent '^ 



Complete this form for as many major activities as you plan to evaluate. 



PpRR 6A 

Responsibility for Data Collection 



?reclsely how does the evaluation systtem work? Two major details of fi\ 
lii^lcmcntlag an evaluation, system include assigning responsibility foV \ 
collecting the^data and designatlng^4:^ection. t^ For each facu lty ' 
activity to be evaluated, fill out^^^^-form listing the required data 
(assune also that faculty may want ^ add smpplenental data), indicate 
yr^i?,''^^'^'^"'^^ for collecting and reporting this information, when 
it will be collected and to wfacmi the data is sent. 



EXAMPLE 

Activity: Advising 
Data 

Advisee evaluation forms 
Retention rates of advisees 

Reports from department heads 



Collected By 

Advisor 

Registrar 



When 



May' 



January & 
September 



Department head May 



To Whom Sent 

Department head 

Department head 
and dean 

Dean 



> 



Activity ; 
Data 



Collected By When ' 



To Whom Sent 



Complete this fonn for as many major activities as you plan to evaluate. 



r FORM 6B . 

Anmuil Calendar of Evaluation Activity 

Making an anmial calendar of activities will provide a display of the 
entire evaluatdLoot systeia and an easy revlev of the -work f lov in the 
process. Devise the da ta for this f omi f rem your coo^lete set of forms 6A« 
After you have displayed all the information, you may want to revise, 
this form axid^some of 6A to achievers better distribution of vork among 
people and the times ofir the year. ' ' . 

One additional activity is also Included in diis form: Analysis of the 
data and action. Do not underestimate the amount of time taken at this 
st.age. For Instance 9 if an all-college conmlt tee Iq charged with evalu- 
ating all faculty members up for tenure ahd promotion, even in a small * 
college they may have to review 25 or 30 people. If a significant amount 
of data has been collected* this proems consumes an enormous amount of 
time* 2e generous in the amount of- time allowed. 



J. T-fme of 

]men Data Collected By To Whom- Sent Analysis & 

. . V- ^ ^ Decisions 



VIII. 

■ 1 



mSTBDMENTS AND PBOCEDlBi£~S, ADOPTION, IMPLEMENTATION, 
EVALOAriON AND FU RT HER REVIEW 



If all of the previous forms have heen completed to your satlsfac- 
tlon, you now. have the basic structure of an evaluation system. Your * 
probable neaet steps are to design specif Ic Itnstrument's and of flee .pro- 
cedures to Implement the system. These details should be worked out 
carefully before the system -Is proposed for adoption. Such a task is 
best done by only a few people who will present the forms and procedures 
to the study group for review and revision. If, for; Instance, the pro- 
^ cedure requires the review, of a variety of student rating instrtjments, 
it may take some time to collect the varloiis instru^nta and assess , ' 
their applicability to your situation. However, some of i^H^actlvlty 
can. take place after the basic system has been adopted If time is a 
critical factor. The Important thing to remember is not to begin using 
any form or procedure before it has been fotmally reviewed and adopted 
by the critical decision-making body — usually the full faculty. We 
have provided no work form to list Instruments and procedures, because 
the variations are infinite. But the study group should be satisfied 
that all details are listed and completed. 

Adoption. Be sure to carefully review the entire study ^irocess 
outlined on Form 1 just prior to adoption to be sure that -all the. 
appropriate steps have been followed. If the process has been open, 
suggestions incorporated into the system and sufficient time . allowed for 
discussion, there should be. llttl« problem having the new system adopted. 

Implementation. OJie study group might helpfully review, implementa- 
tion details with those^who are to- administer the system. The first 'time" 



through may require cpngji aptft ^^ Modification and adjustment. It Is 
important to tentenber t^ in^ the factiltjr generally a? changes in the 
system are being made, even If they are relatively minor. "No surprises" 
Is^a good motto for the first year of^a new evaluation system. 

Evaluation and review > As soon as the evaluation system has been 
run throxigh one complete cycle, some group should report back to the 

entire faculty and appropriate others on how well it has worked. You may 

• ■ t 

want to give some consideration to how this evaluation of the evaluation 
system is to bfe designed before you begin using the system for* the first 
time. The design of this «tudy could be very conflplex if resources are 
ava;llable an^ you n^ed to know how newly-deslgnted forms and processes 
are working, whether they are reliable, and^so^orth. But you may only^ 

* want to know whether or not faculty, stxidents and administrators are 

* 

satisfied with the system. In^tjlther case, you need to be explicit about 
^^lat the criteria for success are. . ^ 

If the new system has been well designed and is clearly acceptable, 
it may nm without substantial revision ^flflf^ several years. It is more • 
likely to run smoothly, however, if a major study Is undertaken after 
two or three years of use. Even if the conclusion of this review is tha^ 
only minor modifications are, necessary. It is well worth doing. No 
system will last forever. Part of the- original legislation adopting a 
system might very well Include a mandated review at some future date. 

This will reassure everyone involved ^hat everything,' even the evalu- 

■f ■ 

atlon system, is to be judged on the basis of merit and not simply 
administrative expediency. * 
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^^ ^^ ^ 

USING STUDENT APPRAISAL INSTRUMENTS 

Too often the major focus of discussions about faculty evaluation 
is the use of student- evaluations or appraisal of Instruction* It is 
\ hard to imagine a college or university that does not utilize some kind 
of student appraisal of instruction as an essential part of a< factaty 
evaluation system. But these instruments are only one source of Infor-* 
aatlon^ and it is unfortunate that their discussion is so highly 'charged 
and political. One good result of all this conversation and concern, 
however 9 is that there has probably been more research and development 
of student rating ih^zisa^ts in recent years than has been lavished on 

any other facet of faculty evaluation. ^ • • ^ 

^' # . 

The questions ;fl6st often asked and. researched regarding student 

> ^ , •* . 

rating instrifments are: "Are these instruments reliable?" and "Are they 
v^id?" In briefs varioiis investigators havei^^und, thab many Instruments 
are reliable (that Is, they are Internally consistent and^ the results 
achieved are stable over time — ori they accurately measure the same 

thing each time they are losed) The validity issue, on the other hand, 

/ 

may never be settled. Validity Is a measure of whether or not an Instru- 

I ' 
ment is evaluating what It is supposed to evaluate. If there is little 

agreement between researchers and users about what Is supposed to be 

measured by stiident rating Instruments, It Is difficult to produce a 

"valid" instrument , . 

• Most people who favor the use of student ratings as an important 

component of an evaluation system claim that they are Intended to measure 
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student reactions to Instruction, and ^e therefore Intrinsically valid. ^ 
Most frequently/ those who oppose the regular use of such Instruments 
argue that they cannot be- valid because the evidence for effective 
teachings Is what students learn, not what students think about what they 
are learning or about what the teadier is doing. ^ We come down on the 
* side of those who claim that 'It is important to know how students' feel 
about lAat' they axp l€amlng^ and to have their reports on what teachers 
are doing when they 'teach. It is also possl^ tcr know what students 
themselves are do'ii^ by the-ifce of stUaent ratings. We openly admit that 
student rating syst^s are, not. valid measures of how much students 
learn. .In fact, there is scant evidence that we can satisfactorily and 
easily measure what students learn in a way that is accepted to every- 
one. Th»t dilemma is one of the major cfeallenges facing educational 
researchers today. * . V 

Assuming that some kind of student "rating Instrument will be used, 
the following general findings about rating instruments might be helpful. 
■1? Students are honest and say^what they believe (as long as the ■ 

instrument is administered in a fair and confidential manner) . 
2. Students are tell ing>ou wh^t they perceive . Their perceptions are 

as real as yours are, even though they may differ. * 



^^^55^(1969?'^439^444 Ratings of y Instruct ion," AAUP Bulletin . 

2 X ' • 
F.N. Kerliiiger, "Student EvalOaticJn of University Professors," School 
and Society . 99 (1971), 353-356. . 
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Rating Instrtnaents tell you vhether the instruction being* delivered 
meets students* perceived needs and meshes with their ^Individual 
learning styles. T^be more specific the questjllon, the better the 
^ i^ormation. For example-, if a student is asked to "tell me three 
things to do to improve my Instruction^" he or she is likely to 
tell you some very specific things that, will be helpful to meet 
that" student * s needs . Those things might not be applicable to other 
stxidents. 

There is some positive weighting, or "halo" effect in ratings 
associated with the age and rank of professors (older, tenured 
professors tend to receive higher ratings) . Those who are experi- 
menting with new approaches to teaching (such as a Rogerian discus^ ^ 
sion technique » or a radically student-centered course design) can' 
expect more negative ratings. Studgnts become anxious if they. are 
on unfamiliar ground* Such ratings should not, of course » discourage 
experimentation* ' ' 

There are some items on which the professors generally rate -so - 
highly that they provide little discrimination among most instruc- 

0 

tors. These include statements indicating that the teacher is 
^rien(i|.y, envoys teaching, has a thorough knowledge of the^^lburse, 
seems kfacfwledgeable, is enthusiastic » maintains good feelings in 
the claAs, likes students to disagree and ask questions, and gives 
individual attention to students. When someone does not score high 
on siich items» should pay attention to these areas. Other ques- 
tions are likely to receive relatively low 'ratings £rom most students 
for most instructors » including questions relating to the use of 
audlovisxx^ jaaterials^ motivation' to take additional related courses » 



the value of laboratory sessions »' the interest and stimulation" of 
reading asslgmients and textbooks, and the student's sense that he 
or ahe Is performing up to his or her potential or has gained a 
better understanding of hlaself /herself through a particular course. 
Hence low ratings here are seldom a cause for alarm. 
Students tend to give helpfxil Information that does differentiate 
between professors on questions that regard general teaching skill, 
classroom interaction, the aaoinlt of feedback given to students, 
the way dlsctisslons are conducted, classroom orgatxlzat Ion , prepara- 
tion, and whether or not the teacher requires a significant amount 
of work for the course. Interestingly, it is usually found that, 
students do not reward a professor because he or she is "easy," 
except when the student already has received a grade that is very 
different from the one the studtot expected. 

Students- do not generally rate the professor in direct correlation 
to the grade the student expects to receive in the course* Often, 
In fact, students who' do very well are harder on their professors 
than students who do poorly. 

Tte,.i^ects of class size, feqxiired and not required courses, 
disciplinary biases and the like can be accommodated by a sensitive 
analysis of rating results. vThe most common idvlce is to compare 
instructors who teach in similar circtimstances with each other. For 



^ James Kullk, "Early Responses from The Campus Use of The Instructor — 
Designed Questionnaire (IDQ),*'" in S.C. Ericksen, "The Lecture," Memo 
to the Faculty . 60 (April, 1978), 5. " 
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instance 9 It is unfair to tompare the ratings of a senior profebsor 
of |^|^robiology> teaching, a research seminar with a Jimlor professor 
teaching a required freshman English course.' 
9. There is sone evidence that student evaluation ^prompts improvement 
in teaching, especially i^en a professor receives ratings that are 
lover than he expected on specific items. 

The process of chooslxig a student appraisal instrument can provide 
a fine occasion for discussions about teaching. Since there are many* 
tiell-desigAed p proven instruments available p it is wise to consider the 
benefits of adoptii^ one of these rather than creating a new instrument 
for yotir ovn> campu^. Designing good instruments is very time consxjming 
and may not be worth the effort as compared with a reasonably good 
system that ±s ready to run. 

i^mong computer-based systems ^ there seems to be greater acceptance 
by faculty of those InstrumiSnts which allow individual professors to 
play some role in customizing the questionnaire p while maintaining a 
core of common questions that can . be x:ompared across departments' and 
disciplines (see, for example p CAFETEJIAp IDEA and IAS below). 

Below are brief descriptions of ^some widely used Instrimients. They 
Illustrate some options currently available. 
1. . "CAFETERIA** type Instruments. \ 

Several years ago the Measurement and Research Center at- 
Purdue University developed a course appraisal system called 
CAFETERIA. This Instrument allows a faculty memker to construct an 
Individualized Inst^ctor and course appraisal consisting of five 
"core" items used by all Instructors and an additional 35 items 
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choscn fro» a catalog of 200 questlona con'cernlng a variety of 
courac and Instructor Issues. The res^lt» are analyzed by computer. 
Including responses to a few "Instructor-supplied Items" hn which 
no norms are available* A norm base has been developed for Purdue 
University and also for a set of Indiana Independent Colleges that 
provides tor comparative percentile scores that are printed on each • 
professor's report In addition to the median responses to each 
itfem. For nofe information write Dr. James Derry. i 
Today there are several other CAFETERIA- type systems In use. 
The complete CAFETERIA package cAn be purchased from the Purdue 
l^BHBk^Poundatlon (ready to be aounced on any computer that can 
accept a contemporary version of FORTRAN). Purdue will also process 
Instruments for some colleges. A "Cadillac" version of CAFETERIA is 
available from the Office of Instructional Resources at the Uni- 
versity of Illinois at Champaign-Urbana 61801 : the Instructor and 
Course- Evaluation System (ICES) allows a variety of reporting 
formats and Information for the Instructor, a catalog of over 600 
items to choose from, and more sophisticated reporting mechanisms. * 
This system uses only three "core'* items: "rate the course content," 
"rate the instructor," and. "rate the course in general". Special 
items are indicated for teaching situations such as science labora- 
tories or field trips, and for various interests the teacher may'' 
have, ranging from warmth and concern for students to the effective 
use of computers.^ • 

A more simplified version. of the CAFETERIA type is the Instructor- 
Designed Questionnaire (IDQ) , available for modification by any 
c&llege through the Center for Research on Learning and Teaching 
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(CRLT), University of Michigan^ A description of this systsn wa» 
published as e Meao to the Feculty by CRLT in 1976, entitled 
**SCudent Resctions to Instruction.** That six-pege monograph in- 
cludes the cc^lete patalog of 153 items that is similar to the one 
from Purdue bu( available without charge and arranged in a somewhat 
mote effective toanner. 

The CAFETERIA approach is used at most of the "Big 10*' unlver- 
sities and a new version for small colleges la being developed by 
the Creat Lpakes Colleges Association Faculty Development Program « 
All of the CAFETERIA variations are most useful for developmental . 
purposes, rather than for summative comparative judgments for 
personnel <fecislons. Hovever, If they are used consistently, over a 
long period of time, the **core*' Items can form a useful basis for 
personnel Judgments if sensitively analyzed. 

Instructional Development and Effectiveness Assessment (IDEA). 

In recent years the IDEA system has been tried by many colleges 
across the country as a result* of a large Kellogg Foundation' gra^it 
for dissemination administered by the Center for Faculty Evaluation 
and Development at K^sas State Univ^sity. This set of instruments 
requires that the professor identify course and learning objectives 
and then the students rate their progress toward these objectives. 

The professors are given suggestions abotlt areas of emphasis which 

■« 

would raise students* responses to various items in the analysis 
report. Cost of Implementing IDEA is relatively high, but services 
from Kansas State University are extensive. Information may be 
obtained from the Center for Faculty Evaluation and Development, 
Post Office Box 30Q0/ Manhattan, Kansas 66502. 



Stndwc Instmetleaal Rcpprt * (SIR) . 

Thl» T«l«tlv*l7 tbort laatroMot !• tb« oldMC «v«llabl« 

I "■■■M 111 sttidflDC appralMl^ It ««k« qoMtions «boot the coorM 
psrtalnlnt to f rhT-»tod<nt rttlatloashlpa, coutm obj^ftctlvea and 
_ orgw ilTT lf m , l«ctnrM, rMdln|^ —tftiiawif, coaCM dlf£lo:ilC7 and 
tauuiaMttaam. Addl tf o n a l ly, •todcnts arc aakad abpdt .th«lr own 
aotlvatlon for taking tha coursa nd axpactad grada. Like tha IDEA 
•ystaa, coat la relatival^ Jilgh, but tba Edocatlonal Teatlng Servlca 
provldaa axcallant procasaisg aazvlcaa for a caapisa tiot vLUlng or 
aqu^ypad to a«t «tp Ita own ayataa. A^allabXe from the Educational 
Teatlng Sarvlcc, Princeton, New Jeraej 08540. 
Teaching Analysla By Studenta (TABS). 

TABS la part of a aore coaprehenalve teaching* laproveaient 
coDMltatloD process .chat vw developed at the foner Clinic to 
I^rove Wverftit7 Trechtpg at the Dnlvereity of Masaechusctts* The 
l&etmeot is' pertlculerly usefxil for classrooa diagnosis and 
follo«r-up measurement of Improvement. A college Intending to use 
TABS should seek consi^ltation from someone fsalilAr with the "clinic 
process to gain its fxill benefits. This system is described fully 
In the Bergquist and Phillips Handbook for Faculty Development , 
Volume II. 

Instructional Assessment System (IAS)* 

/ 

This set of instruments is described in the Bergquist and 
Phillips Handbook for Faculty Development , Volume I. It is a 
collection of seven instruments for different', teaching situations 
stid raters p for example: self-evaluation, peer evaluation, lecture 
courses t studio courses. The materials may be copied directly from ' 
the handbook. No scoring service is available. 
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6. Course Evaluation Questionnaire (CEQ) . 

One of the oldest well-researched computerized Instruments, 
CEQ .contains 23^ "general concept" Items about course attitude. 
Instructional method, course cotxtent, student Interest and the 
instructor. It may be used, now, as part of ICES, described above. 
^Services for using this system are available and more Information 
0-- may be obtained from the Office of Instructional Resources, Measure 
ment and Research Division, 307 Engineering Hall, University of 
Illinois, Urbana, Illinois 61801. _ 

While most of the Instruments described above are relatively easy 
to Install on a computer or score by hand, and* they are all rather good,, 
they are not without "^problems. Often the campus reaction Is negative 
because the Instrument Is simply Ins^all^ without going through* the 
rigorous process of adoption and partlcularization that <^her elements 
Of an evaluation propess have suffeted^ However , m^^ colleges have 
Inadequate time or resources td develop l^faedLr-own. foxias, vhave too few 
l^tructoirs and opportimltry to establish reliable norms^^ and . thus ^houl5 
consdLder the use of a pre-packaged system. Simply remember: campus 
review of the various systems md debate Is'^issential* 

One other caveat: complex rating Instruments, while popular, are ^ 

^' ' " " ■ ' ^ " ' 

usually not the most helpful means 6t gathering da^ to change a course 

plan or make a specific Improvement in teaching'. For instance, asking 

students directly, or in writing, at jnidterm:^ ''What een I do to improve 

this course next week" may be much more useful (if you follow through). 

For a fuller discussloti- of"* the issues involving student appraisal ^ 

Instruments, refer to the following sources: 
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APPENDIX 2 
A CASE FOR DIFFERENTIAL EVALUATION 



Differential evaluation allova different units/departments or even 
individuals to determine the relative Importance or weight of various 
activities to be evaluated. Not only Is this approach different from the 

r 

Standard practice of evaluation by a ccllege-vlde formula, but ±i. requires 

thorough consideration of the philosophical implications of allowing 

Individuals to receive similar rewards (such as promotion and tenure) 

for sometimes significantly .different kinds of activity. Using dlf- 

, » - • ' ' ■ ■ ; 

ferentlal evaluation, differ en t departmecfts could weight scholarly ac- 

tlvlty more heavily than others, or some individual faculty members 

/mlkht be evaluated more heavily fox thelx*^work In student advisement 

tKJitfc would thel^ colleague^. Su^^h an ajgp'roach allows for Individual 

<*iffarences and needs, &mong departments or among faculty within a* 

i department. 

In a hypothetical ^case, South^J^llege has Ifientlfled the following 
actin^ltleis as those to be evaluated and established ranges of weight for 
each: . ^ 



Teaching — 

Advising ' — > » . ^ 

College Service ~ 

Scholarly and Creative Growth 



40-80 percent, average 60 percent 
0-40 percent, average 20 percent 

10^20 percent, average 10 percent 
0-20 percent, average 10 percent 

^ Total: 100 percent 
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Another college might IndLude ^advising as part of teaching or 

service » give nmchniore weight to scholarship^ and consider community 

-* 

service as an additional category* There are obviously many variations 
on this pattern, and sometimes 'one department might have a category that 
no other department In the college has. 

Uslng'.such/^a college-^de guideline , a dean or personnel committee 
would negotiate with each department yearly to Identify variations on 
the college norm necessitated by departmental' needs • Department A, for 
example. Is undergoing extensive curriculum revision and prc^sents a case 
for all of Its faculty to be evaluated . lesS for teaching and more for 
college service, since much energy and time woyld be devoted to ^he 



currlcular 


revlsloij 


I. 










/ 


Activity 


College 
Norm 


Dept^ A 
vAll Faculty) (Prftf. 


A) 


Dept. 
(Prof. 


B 

B) 


(Prof. C) 


Teaching 


60% 


. s 50Z 


\ 




AOZ 




602 


Advising 


2pZ 


20Z 






402 




. 20Z * 


> 

College 
















Service 


lOZ 


20Z 


lOZ 




202 




02 


Scholarly 
















Growth 


102 


lOZ 


LOZ 




02 




20-2 


Totals : 


lOOX 


lOOZ 


lOOZ" 




1002 


9 


1002 



Dept. B. 



602 

2Qit« 
A 

10% 



lOZ 



100% 



In the case of Department B the chair wished to take Into account 

k 

Individual differences In clrcximstances among the faculty, so she pro- ' 
posed a differential evaluation among Individuals. When averaged, these 
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dlf femcM confonaed to the college norm (tliough this need not necea* 
•arlly be the caae)^. Profeaaor A teachea well, but his' advising skills 
are ao bad that he has been relieved of that responsibility (and he has 
Indicated that sometline' during the year he will attend some advising 
skills workshops). Professor B^^m excellent advisor, has her teaching 
load reduced so that she may handle more advising; she also serves on 
many committees and is willing to take no credit for scholarly growth 
this year. Professor C has been relieved of college-wide commitments so 
that hft can complete his dissertation this year. In this hypothetical 
case, the dean assumed that the college would allow individual variations 
within the department pnly if the average total for the department came 
very close to the college norm so that departments would ^lot veer too 
sharply from each other in their performance. 

Differential evaluation requires that college-wide norms be develo 
and that deans or department heads negotiate xjlth their faculty and 
ful^ consider both individual and departmental needs each year. Such a 
pjoce^s is particularly helpful JLf a college or department begins to * ^ 
exper imen t with growth contracts ^ (see Appendix 3J , but is also a w^y to# 
work out the natural differences between work assignments among 4hK>art-T ' 
ments that occur in almost every college (such as the differences betwe^* 

- ■ - ■ . ■ } 

the English Department, Physical Education and Music). 
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APPENDIX 3 
GROWTH CONTRACTS 



..One of thm newer approaches to faculty evaluation and development 
1> the growth contract. Zxanples of Institutions already using growth 
' contracts are Gordon College^ Mew College at the ^University of Alabama, 
Ottawa University, Los Medanos College, Columbus College and Austin 
College. Here Is a process by which Individual faculty assess their 
professional strengths and those areas they Wish to^ Improve. They write 
a growth contract or plan which la negotiated or shared with colleagues, 
the department bead, the dean or a committee. The contract may dLnclude a 
stipulation of what resources, time or othef support the Institution 
vlll provide the IndlvldtiaL to meet the goals. Each year progress on the 
growth contract Is evaluated. The major evaluation usu^ly occurs at a 
midpoint assessment stage apd secondarily at the end of the contract. 

Colleges differ^ widely In the way growth contracts are utilized. 
Some u^^ them Informally as .a supplement to^sumraatlve evaluation; for 
others.lt Is the only basis for evaluation. Some require negotiations 
with the department head or dean; others encourage sharing with col- 
leagues. Some employ a convex system of committees for each contract; 
others use no formal supervision. Others have offered personal growth or 
llfe-^laxmlng workshops which culminate In writing of growtft plans which 
may or may not be "contractual" with the Institution. 

*The Important points about growth. contracts are that they are 
developmental, that they allow the Individual to determine what will be 
evaluated,, that they force Consideration, of evaluation at the bieglnnlng 
of a,ye4u: or period of years, and that they are relatively easy to 
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evaluate. It is not difficult to aas^ps whether or not the person did 

vhat Ym or she said was going to be done «e opposed to assessing vhether 

or not the faculty ntember Ls a good teacher or meritorious member of the 

conmunlty. Major problems ^vlth growth contracts are that It. la difficult 

to establlith reasonable expectations the first few times and It Is 

difficult to achieve comparability between Individuals' growth. 

For further Information, 6ee; 

Buhl» Lance C. and Greenfield , Ad^le. "Contracting for Professional 
Developoient In Academe." Educational Record , 56, (Spring 1975), 
•111-121 • ' 

Gross, Richard, "Faculty Gro*fth Contracts:" Educational Horizons (Winter 
1976), 74-79. ' ~ 

Hodgkinson, Harold L. "Faculty Reward and Assessment Systems," la B.L. 
Smith., ed.. The Tenure Debaty . San Francisco: Jossey-Bass, 1973. 




